In video systems where graphics are added to video content, there may be a video layer and a graphics layer on top of the video layer. The video layer may come from a television (TV) source, and the graphics layer may be used to add information such as, for example, a user guide or some kind of a user-graphical interface such as a TV menu. In more sophisticated video systems, there may be graphics added on top of the video, as well as, graphics below the video. For example, adding a user guide on top of the video and a still background behind the video.
Outputting the layers as one output onto a display such as, for example, a monitor, requires blending the layers into one stream of data. The process of blending the layers together is known as graphics blending or graphics and video compositing.
A system with graphics and video layers may be viewed as a planar system with planes composed of the different layers. A low-end system consists of only one graphics layer and one video layer. A higher-end system consists of at least one graphics layer above the video layer, and at least one graphics layer below the video layer.
The way to blend the layers is generally done from the bottom up. So in a system with three layers, for example, the blending is done by blending the bottom graphics layer with the video layer, which results in a new blended layer, which may then be blended with the top graphics layer to get a composite image to output to the display.
Each layer of graphics, or the layer of video is composed of a buffer and an alpha. The buffer is a region of the memory in the system that contains pixels of the layer. Alpha is the blend factor of a layer and it indicates how much to blend of that layer with the layer below it. The value of alpha ranges from 0 to 1, inclusive. For video there alpha value can be the same for the whole layer or per pixel. Whereas with graphics, each pixel may have a different alpha value.
For example, if a system has two layers, a graphics layer on the top and a video layer below it, the buffer and the alpha for the graphics and the video layer would be, Bt, At, Bb, and Abs, respectively. Blending the two layers together yields the following:Btb=AtBt+(1−At)Bb  (1)Where Btb is the buffer for the blended layer. If At is 1, then when blending the graphics with the video below it, all that is seen is entirely graphics, so an alpha value of 0 implies complete transparency. If At is 0, then when blending the graphics with the video below it, all that is seen is entirely video, so an alpha value of 1 implies complete opaqueness.
In most systems, alpha is an 8-bit number ranging from 0 to 1, inclusive. So there are 256 levels of transparency ranging from complete transparency to complete opaqueness.
In a more complex system, with graphics layers above and below the video layer, things may get more complex as well. For example, a system may have a graphics layer on top of the video layer, with buffer B1, and alpha A1, the video layer with buffer BV and alpha AV, and a graphics layer below the video layer, with buffer B2 and alpha A2. Applying equation (1) above, blending the video and the graphics layer below it yields:BV2=AVBV+(1−AV)B2  (2)Where BV2 is the buffer for the blended bottom layer. Then blending the top graphics layer with the blended bottom layer yields:BV3=A1B1+(1−A1)BV2  (3)Where BV3 is the buffer for the three blended layers. Expanding and re-arranging equation (3) after applying equation (2) yields:BV3=AV(1−A1)BV+A1B1+(1−A1)(1−AV)B2  (4)
Equation (4) above illustrates the calculation required to blend two layers of graphics with one layer of video. In more complex systems, there may be several layers of graphics above a layer of video, and several layers of graphics below the layer of video. In such systems, the graphics layers on top may be blended together into one top graphics layer, the graphics layers below may be blended together into one bottom graphics layer, then the top layer, video layer, and bottom layer, may be blended together according to equation (4).
In video systems, hardware that performs the calculations for compositing needs to read all three layers simultaneously, with the video, which is streaming data, and output the result onto the monitor in real-time, which can get bandwidth-expensive. The compositing process can also be hardware expensive when two graphics layers are read out of the memory, and calculations are made to accommodate streaming video data. Problems may be seen sometimes on a personal computer (PC), for example, when moving a window, a portion of the screen that was covered by the window may remain blank for a few seconds, because the graphics engine may take time to respond and do all the blending to accommodate the new graphics layers.
Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of ordinary skill in the art through comparison of such systems with the present invention.